The Role of Knowledge-based Features in Polarity Classification at Sentence Level

Though polarity classification has been extensively explored at document level, there has been little work investigating feature design at sentence level. Due to the small number of words within a sentence, polarity classification at sentence level differs substantially from document-level classification in that resulting bag-of-words feature vectors tend to be very sparse resulting in a lower classification accuracy. In this paper, we show that performance can be improved by adding features specifically designed for sentence-level polarity classification. We consider both explicit polarity information and various linguistic features. A great proportion of the improvement that can be obtained by using polarity information can also be achieved by using a set of simple domainindependent linguistic features. Introduction One of the most popular subtasks of opinion mining is polarity classification, i.e. the task of distinguishing between positive and negative utterances. This task has been extensively explored at document level but there has only been comparatively little work at sentence level although the task is an established research problem (Matsumoto, Takamura, and Okumura 2005; Meena and Prabhabkar 2007). Sentiment information is not evenly distributed across a document. Not only do documents usually comprise both subjective and factual sentences but also the polarity of subjective sentences within a document varies. Thus, sentencelevel classification can be used to improve document-level classification (McDonald et al. 2007). Moreover, for tasks demanding fine-grained text analyses, such as text summarization, sentiment classification at sentence level seems more appropriate than document classification. Due to the small number of words within a sentence, polarity classification at sentence level differs substantially from document-level classification in that resulting feature vectors encoding sentences tend to be much sparser. Therefore, a classifier trained on bag of words performs worse than at document level. Fortunately, there is a plethora of linguistic features by which a word can be described within a sentence. We consider features, such as part-of-speech information, clause Copyright © 2009, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. types, depth of word constituents, or WordNet hypernyms. At document level, these features have hardly been used. In general, the benefit of these features remains controversial since their extraction is computationally expensive (many of these features require linguistic pre-processing such as part-of-speech tagging or even syntactic parsing) and their contribution in terms of performance is fairly limited since bag-of-words classifiers already pose a robust baseline. We show that explicit polarity information and a set of simple linguistic features can significantly improve a standard bag-of-words classifier. The additional insight that a standard classifier can be improved by linguistic features in the absence of any polarity information might be useful for situations in which no domain knowledge is available since polarity information is domain-dependent to a great extent. We consider polarity classification as a binary classification task. That is we assume that each sentence to be classified is subjective. We neglect the distinction between objective and subjective content since this classification is usually solved independently (Pang and Lee 2004; Ng, Dasgupta, and Arifin 2006). Our experiments are carried out on a subset of the MPQA corpus (Wiebe, Wilson, and Cardie 2003).